

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>RGW Data caching and CDN &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script data-url_root="../../" id="documentation_options" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/doctools.js"></script>
        <script src="../../_static/sphinx_highlight.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="Ceph 对象网关 Swift API" href="../swift/" />
    <link rel="prev" title="Ceph Object Gateway IAM API" href="../iam/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../" class="icon icon-home" aria-label="Home"></a></li>
          <li class="breadcrumb-item"><a href="../">Ceph 对象网关</a></li>
      <li class="breadcrumb-item active">RGW Data caching and CDN</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../_sources/radosgw/rgw-cache.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../" class="icon icon-home"> Ceph
          

          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephadm/">Cephadm</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Ceph 对象网关</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../frontends/">HTTP 前端</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multisite/">多站配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../zone-features/">域的功能</a></li>
<li class="toctree-l2"><a class="reference internal" href="../placement/">存储池归置与存储类</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multisite-sync-policy/">多站同步策略配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../pools/">存储池的配置</a></li>
<li class="toctree-l2"><a class="reference internal" href="../config-ref/">配置参考</a></li>
<li class="toctree-l2"><a class="reference internal" href="../admin/">管理指南</a></li>
<li class="toctree-l2"><a class="reference internal" href="../account/">用户账户</a></li>
<li class="toctree-l2"><a class="reference internal" href="../s3/">S3 API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../iam/">IAM API</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">数据缓存和 CDN</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#new-apis">New APIs</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#auth-api">Auth API</a></li>
<li class="toctree-l4"><a class="reference internal" href="#cache-api">Cache API</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#using-nginx-with-rgw">Using Nginx with RGW</a></li>
<li class="toctree-l3"><a class="reference internal" href="#appendix">Appendix</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../swift/">Swift API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../adminops/">管理操作 API</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/">Python 接口</a></li>
<li class="toctree-l2"><a class="reference internal" href="../nfs/">通过 NFS 导出</a></li>
<li class="toctree-l2"><a class="reference internal" href="../keystone/">与 OpenStack Keystone 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../barbican/">与 OpenStack Barbican 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../vault/">与 HashiCorp Vault 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../kmip/">与 KMIP 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../opa/">与 Open Policy Agent 对接</a></li>
<li class="toctree-l2"><a class="reference internal" href="../multitenancy/">多租户</a></li>
<li class="toctree-l2"><a class="reference internal" href="../compression/">压缩</a></li>
<li class="toctree-l2"><a class="reference internal" href="../ldap-auth/">LDAP 认证</a></li>
<li class="toctree-l2"><a class="reference internal" href="../encryption/">服务器端加密</a></li>
<li class="toctree-l2"><a class="reference internal" href="../bucketpolicy/">桶策略</a></li>
<li class="toctree-l2"><a class="reference internal" href="../dynamicresharding/">动态的桶索引重分片</a></li>
<li class="toctree-l2"><a class="reference internal" href="../mfa/">多因子认证</a></li>
<li class="toctree-l2"><a class="reference internal" href="../sync-modules/">同步模块</a></li>
<li class="toctree-l2"><a class="reference internal" href="../notifications/">Bucket Notifications</a></li>
<li class="toctree-l2"><a class="reference internal" href="../layout/">RADOS 中的数据布局</a></li>
<li class="toctree-l2"><a class="reference internal" href="../STS/">STS</a></li>
<li class="toctree-l2"><a class="reference internal" href="../STSLite/">STS Lite</a></li>
<li class="toctree-l2"><a class="reference internal" href="../keycloak/">Keycloak</a></li>
<li class="toctree-l2"><a class="reference internal" href="../session-tags/">Session Tags</a></li>
<li class="toctree-l2"><a class="reference internal" href="../role/">Role</a></li>
<li class="toctree-l2"><a class="reference internal" href="../orphans/">Orphan List and Associated Tooliing</a></li>
<li class="toctree-l2"><a class="reference internal" href="../oidc/">OpenID Connect Provider</a></li>
<li class="toctree-l2"><a class="reference internal" href="../troubleshooting/">故障排除</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../man/8/radosgw/">radosgw 手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../man/8/radosgw-admin/">radosgw-admin 手册页</a></li>
<li class="toctree-l2"><a class="reference internal" href="../qat-accel/">使用 QAT 为加密和压缩提速</a></li>
<li class="toctree-l2"><a class="reference internal" href="../s3select/">S3-select</a></li>
<li class="toctree-l2"><a class="reference internal" href="../lua-scripting/">Lua Scripting</a></li>
<li class="toctree-l2"><a class="reference internal" href="../d3n_datacache/">D3N Data Cache</a></li>
<li class="toctree-l2"><a class="reference internal" href="../cloud-transition/">Cloud Transition</a></li>
<li class="toctree-l2"><a class="reference internal" href="../metrics/">Metrics</a></li>
<li class="toctree-l2"><a class="reference internal" href="../uadk-accel/">UADK Acceleration for Compression</a></li>
<li class="toctree-l2"><a class="reference internal" href="../bucket_logging/">桶的日志记录</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../monitoring/">监控概览</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../hardware-monitoring/">硬件监控</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <section id="rgw-data-caching-and-cdn">
<h1><a class="toc-backref" href="#id1" role="doc-backlink">RGW Data caching and CDN</a><a class="headerlink" href="#rgw-data-caching-and-cdn" title="Permalink to this heading"></a></h1>
<div class="versionadded">
<p><span class="versionmodified added">New in version Octopus.</span></p>
</div>
<nav class="contents" id="contents">
<p class="topic-title">Contents</p>
<ul class="simple">
<li><p><a class="reference internal" href="#rgw-data-caching-and-cdn" id="id1">RGW Data caching and CDN</a></p>
<ul>
<li><p><a class="reference internal" href="#new-apis" id="id2">New APIs</a></p>
<ul>
<li><p><a class="reference internal" href="#auth-api" id="id3">Auth API</a></p></li>
<li><p><a class="reference internal" href="#cache-api" id="id4">Cache API</a></p></li>
</ul>
</li>
<li><p><a class="reference internal" href="#using-nginx-with-rgw" id="id5">Using Nginx with RGW</a></p></li>
<li><p><a class="reference internal" href="#appendix" id="id6">Appendix</a></p></li>
</ul>
</li>
</ul>
</nav>
<p>This feature adds to RGW the ability to securely cache objects and offload
the workload from the cluster, using Nginx.</p>
<p>After an object is accessed the first time it will be stored in the Nginx
cache directory. When data is already cached, it need not be fetched from RGW.
A permission check will be made against RGW to ensure the requesting user
has access. This feature is based on some Nginx modules, ngx_http_auth_request_module,
<a class="reference external" href="https://github.com/kaltura/nginx-aws-auth-module">ngx_http_aws_auth</a>,
Openresty for Lua capabilities.</p>
<p>Currently, this feature will cache only AWSv4 requests (only s3 requests),
caching-in the output of the 1st GET request and caching-out on subsequent
GET requests, passing thru transparently PUT, POST, HEAD, DELETE and COPY
requests.</p>
<p>The feature introduces 2 new APIs: Auth and Cache.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The <a class="reference external" href="../d3n_datacache/">D3N RGW Data Cache</a> is an alternative data caching mechanism
implemented natively in the Rados Gateway.</p>
</div>
<section id="new-apis">
<h2><a class="toc-backref" href="#id2" role="doc-backlink">New APIs</a><a class="headerlink" href="#new-apis" title="Permalink to this heading"></a></h2>
<p>There are 2 new APIs for this feature:</p>
<p>Auth API - The cache uses this to validate that a user can access the
cached data.</p>
<p>Cache API - Adds the ability to securely override the Range header, that way
Nginx can use its own <a class="reference external" href="https://www.nginx.com/blog/smart-efficient-byte-range-caching-nginx/">smart cache</a> on top of S3. Using this API gives
the ability to read ahead objects when clients asking a specific range
from the object. On subsequent accesses to the cached object,
Nginx will satisfy requests for already-cached ranges from the cache.
Uncached ranges will be read from RGW (and cached).</p>
<section id="auth-api">
<h3><a class="toc-backref" href="#id3" role="doc-backlink">Auth API</a><a class="headerlink" href="#auth-api" title="Permalink to this heading"></a></h3>
<p>This API Validates a specific authenticated access being made to the cache,
using RGW’s knowledge of the client credentials and stored access policy.
Returns success if the encapsulated request would be granted.</p>
</section>
<section id="cache-api">
<h3><a class="toc-backref" href="#id4" role="doc-backlink">Cache API</a><a class="headerlink" href="#cache-api" title="Permalink to this heading"></a></h3>
<p>This API is meant to allow changing signed Range headers using a privileged
user, cache user.</p>
<p>Creating cache user:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ radosgw-admin user create --uid=&lt;uid for cache user&gt; --display-name=&quot;cache user&quot; --caps=&quot;amz-cache=read&quot;
</pre></div>
</div>
<p>This user can send to the RGW the Cache API header <code class="docutils literal notranslate"><span class="pre">X-Amz-Cache</span></code>, this header
contains the headers from the original request(before changing the Range header).
It means that <code class="docutils literal notranslate"><span class="pre">X-Amz-Cache</span></code> built from several headers.
The headers that are building the <code class="docutils literal notranslate"><span class="pre">X-Amz-Cache</span></code> header are separated by
char with ASCII code 177 and the header name and value are separated by
char ASCII code 178.
The RGW will check that the cache user is an authorized user and if it is
a cache user, if yes it will use the <code class="docutils literal notranslate"><span class="pre">X-Amz-Cache</span></code> to revalidate that
the user has permissions, using the headers from the X-Amz-Cache.
During this flow, the RGW will override the Range header.</p>
</section>
</section>
<section id="using-nginx-with-rgw">
<h2><a class="toc-backref" href="#id5" role="doc-backlink">Using Nginx with RGW</a><a class="headerlink" href="#using-nginx-with-rgw" title="Permalink to this heading"></a></h2>
<p>Download the source of Openresty:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ wget https://openresty.org/download/openresty-1.15.8.3.tar.gz
</pre></div>
</div>
<p>git clone the AWS auth Nginx module:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ git clone https://github.com/kaltura/nginx-aws-auth-module
</pre></div>
</div>
<p>untar the openresty package:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ tar xvzf openresty-1.15.8.3.tar.gz
$ cd openresty-1.15.8.3
</pre></div>
</div>
<p>Compile openresty, Make sure that you have pcre lib and openssl lib:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ sudo yum install pcre-devel openssl-devel gcc curl zlib-devel nginx
$ ./configure --add-module=&lt;the nginx-aws-auth-module dir&gt; --with-http_auth_request_module --with-http_slice_module --conf-path=/etc/nginx/nginx.conf
$ gmake -j $(nproc)
$ sudo gmake install
$ sudo ln -sf /usr/local/openresty/bin/openresty /usr/bin/nginx
</pre></div>
</div>
<p>Put in-place your Nginx configuration files and edit them according to
your environment:</p>
<blockquote>
<div><ul class="simple">
<li><p>All Nginx conf files are under: <a class="reference external" href="https://github.com/ceph/ceph/tree/main/examples/rgw-cache">https://github.com/ceph/ceph/tree/main/examples/rgw-cache</a></p></li>
<li><p><cite>nginx.conf</cite> should go to <cite>/etc/nginx/nginx.conf</cite></p></li>
<li><p><cite>nginx-lua-file.lua</cite> should go to <cite>/etc/nginx/nginx-lua-file.lua</cite></p></li>
<li><p><cite>nginx-default.conf</cite> should go to <cite>/etc/nginx/conf.d/nginx-default.conf</cite></p></li>
</ul>
</div></blockquote>
<p>The parameters that are most likely to require adjustment according to
the environment are located in the file <cite>nginx-default.conf</cite></p>
<p>Modify the example values of <em>proxy_cache_path</em> and <em>max_size</em> at:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">proxy_cache_path</span> <span class="o">/</span><span class="n">data</span><span class="o">/</span><span class="n">cache</span> <span class="n">levels</span><span class="o">=</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span><span class="p">:</span><span class="mi">2</span> <span class="n">keys_zone</span><span class="o">=</span><span class="n">mycache</span><span class="p">:</span><span class="mi">999</span><span class="n">m</span> <span class="n">max_size</span><span class="o">=</span><span class="mi">20</span><span class="n">G</span> <span class="n">inactive</span><span class="o">=</span><span class="mi">1</span><span class="n">d</span> <span class="n">use_temp_path</span><span class="o">=</span><span class="n">off</span><span class="p">;</span>
</pre></div>
</div>
<p>And modify the example <em>server</em> values to point to the RGWs URIs:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">server</span> <span class="n">rgw1</span><span class="p">:</span><span class="mi">8000</span> <span class="n">max_fails</span><span class="o">=</span><span class="mi">2</span> <span class="n">fail_timeout</span><span class="o">=</span><span class="mi">5</span><span class="n">s</span><span class="p">;</span>
<span class="n">server</span> <span class="n">rgw2</span><span class="p">:</span><span class="mi">8000</span> <span class="n">max_fails</span><span class="o">=</span><span class="mi">2</span> <span class="n">fail_timeout</span><span class="o">=</span><span class="mi">5</span><span class="n">s</span><span class="p">;</span>
<span class="n">server</span> <span class="n">rgw3</span><span class="p">:</span><span class="mi">8000</span> <span class="n">max_fails</span><span class="o">=</span><span class="mi">2</span> <span class="n">fail_timeout</span><span class="o">=</span><span class="mi">5</span><span class="n">s</span><span class="p">;</span>
</pre></div>
</div>
<p>It is important to substitute the <em>access key</em> and <em>secret key</em> located
in the <cite>nginx.conf</cite> with those belong to the user with the <cite>amz-cache</cite> caps.
For example, create the <cite>cache</cite> user as following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">radosgw</span><span class="o">-</span><span class="n">admin</span> <span class="n">user</span> <span class="n">create</span> <span class="o">--</span><span class="n">uid</span><span class="o">=</span><span class="n">cacheuser</span> <span class="o">--</span><span class="n">display</span><span class="o">-</span><span class="n">name</span><span class="o">=</span><span class="s2">&quot;cache user&quot;</span> <span class="o">--</span><span class="n">caps</span><span class="o">=</span><span class="s2">&quot;amz-cache=read&quot;</span> <span class="o">--</span><span class="n">access</span><span class="o">-</span><span class="n">key</span> <span class="o">&lt;</span><span class="n">access</span><span class="o">&gt;</span> <span class="o">--</span><span class="n">secret</span> <span class="o">&lt;</span><span class="n">secret</span><span class="o">&gt;</span>
</pre></div>
</div>
<p>It is possible to use Nginx slicing which is a better method for streaming purposes.</p>
<p>For using slice you should use <cite>nginx-slicing.conf</cite> and not <cite>nginx-default.conf</cite></p>
<p>Further information about <a class="reference external" href="https://docs.nginx.com/nginx/admin-guide/content-cache/content-caching/#byte-range-caching">Nginx slicing</a>.</p>
<p>If you do not want to use the prefetch caching, it is possible to replace
<cite>nginx-default.conf</cite> with <cite>nginx-noprefetch.conf</cite>. Using <cite>noprefetch</cite> means
that if the client is sending range requests of 0-4095 and then 0-4096,
Nginx will cache those requests separately, So it will need to fetch
those requests twice.</p>
<p>Run Nginx (openresty):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ sudo systemctl restart nginx
</pre></div>
</div>
</section>
<section id="appendix">
<h2><a class="toc-backref" href="#id6" role="doc-backlink">Appendix</a><a class="headerlink" href="#appendix" title="Permalink to this heading"></a></h2>
<p><strong>A note about performance:</strong> In certain instances like development
environment, disabling the authentication by commenting the following line
in <cite>nginx-default.conf</cite>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1">#auth_request /authentication;</span>
</pre></div>
</div>
<p>may (depending on the hardware) increases the performance significantly as
it forgoes the auth API calls to radosgw.</p>
</section>
</section>



<div id="support-the-ceph-foundation" class="admonition note">
  <p class="first admonition-title">Brought to you by the Ceph Foundation</p>
  <p class="last">The Ceph Documentation is a community resource funded and hosted by the non-profit <a href="https://ceph.io/en/foundation/">Ceph Foundation</a>. If you would like to support this and our other efforts, please consider <a href="https://ceph.io/en/foundation/join/">joining now</a>.</p>
</div>


           </div>
           
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="../iam/" class="btn btn-neutral float-left" title="Ceph Object Gateway IAM API" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="../swift/" class="btn btn-neutral float-right" title="Ceph 对象网关 Swift API" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).</p>
  </div>

   

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>